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Abstract 


This article investigates the appearance of robots on stage in contemporary performance. As a convincing 
demonstration of the relevance of robots — as artefact, concept and metaphor — we take Blanca Li’s 2013 dance 
performance Robot! as our starting point. We have distilled five dimensions that provide vocabulary and 


terminology to characterise different instances of robots on stage with a focus on their expressive character. 
These dimensions are: Movements, Apparent level of Autonomy & Interaction, Appearance, Massification and 
Sound 
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1. Introduction 


25 minutes into the contemporary dance performance 
Robot! the show’s protagonist — a 58 cm tall, plastic 
robot of the label Nao - is introduced as it teams 
up with a male, human dancer in a passionate Pas- 
de-deux. The Nao is brought on stage in a closed 
box by the solo dancer: Newly bought - still to be 
unpacked from its box. New-born - with its visual 
resemblance to a child, about to explore how to move 
and come into being. During the sequence, the Nao 
slowly unfolds into a human-like character as it learns 
to dance; see Fig. 1. Being interested in the use of 
robots on stage, this is a great example of a performance 
that pushes the limits of the performative potential of 
the Nao robot: Choreographing a compelling human- 
robot-interaction. Projecting feelings and empathy into 
the plastic robot. Making the audience go “aahh!” in 


sympathy with the animate creature on stage. Figure 1. Rehearsing the Pas-des-deux of Blanca Li's Robot! 
Interestingly, and perhaps surprisingly, this before performing in Macon, 2016. 
sequence’s convincing interaction is entirely pre- (Photo copyright © Henning Christiansen 2016) 


recorded. This sequence has been developed in an 
iterative process with choreographer Blanca Li in the 
role of the robot, and subsequently transferring her 
movements and gestures to the robot and testing with emotional and communicative cues towards which 
the dancer step by step. During the live performance, the dancer reacts in real time. And yet the scene 


the robot replays these patterns and their embedded convincingly appears as a tender, mutually emphatic 
and, to some extent, spontaneous interplay between 


two equal partners, the robot and the male dancer. 
*Corresponding authors. Email: henning@ruc.dk, lindelof@ruc.dk According to engineer Thomas Pachoud and assisting 
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choreographer Pascale Peladan (personal conversation), 
this was the only meaningful way to create this 
sequence due to both the imperfections and limitations 
of (the Nao) robot technology and the detailed timing 
and flow of the show that do not leave a fraction of 
a second for the robot to wander off. Any interest 
in robots on stage is confronted with this dilemma 
between working with the fascinating, autonomous 
behaviour of robots and the demands of control in 
show business. 

This article proposes a vocabulary and an ontology 
to characterise robots’ appearances in scenic perfor- 
mances. In doing so, we take a dramaturgic point of 
view on robots on stage. Focusing on dramaturgy — as 
opposed to using a technical definition of what is a 
robot — means that we consider how a notion of robot 
is experienced, and if and how autonomy is enacted on 
stage. This aim is inspired by the increasing number of 
robots appearing in staged performances, the intriguing 
nature of the very subject, and further motivated by 
our own work in progress design of animating familiar 
domestic objects for stage performances. The article 
takes its starting point in Robot ! which exposes a highly 
diverse array of staged, robotic gestalts. 

Robot! is a dance performance by Bianca Li Dance 
Company featuring dancing robots and human dancers, 
accompanied by an on-stage mechanical orchestra. It 
plays with the ubiquitous presence of technology in 
our society and the way in which robot technology has 
gradually moved from research labs and specialised 
industrial plants. They have become an integral part of 
people’s daily lives in the shape of intelligent copiers, 
lawn movers, vacuum cleaners, toys, parking assistants, 
and a horde of self-driving vehicles now readily rolling 
in. This “uncanny ballet with men and machines,” 
as described by the magazine Elle! stages human- 
robot relationships through choreographic means of 
expression. Featuring a dance company of eight highly 
skilled dancers and the Nao-robot with its specific 
technical qualities and anthropomorphic character as 
a main figure, the show allows for subtle explorations 
of the relationship between robots and humans across 
its various tableaux. Complementing what we might 
coin the “main story” about a Nao robot learning 
to dance, various other types of robotic appearance 
are on stage during the show, including robot 
vacuum cleaners, remote controlled moving vehicles, 
mechanical musicians and humans dressed as cartoon 
robots. While this variety of appearances all refer to 
some notion of a robot, they differ with regard to their 
robotic gestalt and their grade of autonomy, pointing 


LELLE, December 13, 2013. Interview with Blanca Li with comments 
by Sylvia Jorif: “... étrange spectacle ballet entre des hommes et des 
machine”. 
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towards an ambiguity in what we perceive as robots on 
stage. 

The question of autonomy is central to any discussion 
of robots on — or off — stage. A narrow definition 
of robots would, as [24] suggest, imply that “the 
exigencies of live performance require robots to 
move autonomously or semi-autonomously alongside 
human performers and in coordination with human 
operators”. In order to characterise performing robots 
from a technical implementation point of view, Lu 
(2012) suggests a two-dimensional matrix model whose 
dimensions are Autonomy and Control. For example, 
autonomy ranges from the robot being entirely 
controlled by a human to the robot controlled by 
an algorithm (e.g., based on artificial intelligence). 
This view of what a robot is becomes too narrow 
for our purpose as the Robot! performance clearly 
illustrates. Accepting the show’s many notions of 
a robot on stage and considering the (intended) 
audience response, we suggest pragmatic as well 
as dramaturgical reasons for a broader definition. 
We put forward a many-dimensional matrix and 
metaphorical vector-space in which each point indicates 
one particular form of appearance of the notion of 
“a robot”. These dimensions are intended to represent 
essential, identifiable properties which are more or less 
orthogonal and all together span a rich and complete 
space of “robotic” performative opportunities. The 
application of this matrix is double, serving as a tool for 
analytical perspectives as well as for explorations into 
the design of robots for the stage. 


Outline 


Emphasising the cross-disciplinary interests when 
studying robots on stage, Section 2 gives an overview 
of related studies of performing artefacts, focusing 
in particular on the fields of performance studies 
and Human Robot Interaction (HRI). Section 3 
narrows down our focus towards robots on stage as 
“performative gestalts” and includes some remarks on 
how we see the state of the art in robot technology as 
relevant for the present study. In Section 4, we present 
our classification matrix and the five dimensions that 
we have identified on the basis of Robot! and other 
stage performances; we describe each one in more detail 
and eventually exemplify the dimensions in use. The 
concluding Section 5 summarises the argument and 
discusses further issues and perspectives for future 
work. 


2. Studies of performing artefacts 


The present study touches upon many different 
research areas, ranging from robot technology and 
engineering to performance theory and philosophical 
treatments of artificial intelligence. We do not intend 
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to give a complete or representative survey of the 
vast literature. Instead, we wish to provide as much 
background as is necessary for contextualising the 
present study and for making our own starting point 
clear. 


2.1. Robots in performance and in performance 
studies 


As robots in recent years have become increasingly 
cheaper, more robust and easier to program, they have 
also become increasingly apparent on stage. Not just 
on podiums, at fairs, or in RoboCups, but also as part 
of more traditional artistic practices like contemporary 
dance, popular music performance and naturalistic the- 
atre. Industry robots are doing synchronised dancing in 
popular music shows such as Mylene Farmer’s Concert 
Tour (2013) and the Eurovision Song Contest’s semi- 
final (2016). Androids appear as protagonists in nat- 
uralistic theatre performances in Hirata Oriza’s Robot 
Theatre (2004-), robotic objects as dancing props with 
agency (e.g., quadcopters in Sparked by the Cirque du 
Soleil, 2014), and a variety of types of robots in dance 
and theatre performances such as Robot! by Bianca 
Li, 2013 and School of Moon by Eric Minh Cuong 
Castaing, 2016), in opera (e.g., Death and the Powers, 
2010, developed at MITmediaLab and My Square Lady, 
by the artist collective Gob Squad) as well as in per- 
formance installations (e.g., fish-bird-series 2003-2009, 
by Mari Velonaki). All of these performances include 
robots in their set-up, but differ in their emphasis 
on spoken dialogue, movement or sound, and with 
regard to technological means as well as to dramaturgic 
purpose. Also, performance has always adopted and 
played with new technologies. Certainly, including life- 
like artefacts in performance is not a new phenomenon. 
We shall here bypass habitual references to ancient 
Greece and the early stages of automatons and jump 
to the early industrial age as the background of some 
of the basic thoughts on human-machine interaction 
we still identify today, thematically centred around the 
question whether or not machines may replace “the 
real”. We see well-known examples in the mid 19th 
century, such as Hans Christian Andersen’s fairytale 
The Nightingale* from 1843 and the mechanical dancing 
puppet Olympia in Jaques Offenbach’s opera The Tales 
of Hoffmann? from 1881. The issue in The Nightingale 
is the experience of a mechanical bird, from the first 
fascination of its perfection in singing technique, to 
the boredom and depression due to its lack of heart 


? Original title Nattergalen. Original text at 
http://adl.dk; search for the title. English translation at 
http://andersen.sdu.dk/vaerk/hersholt/TheNightingale.html. 
3Original title Les contes d’Hoffmann; original libretto and English 
translation at http://www.gutenberg.org/ebooks/15915. 
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and variation, and ultimately to the acclaim of the 
real nightingale and its song. Olympia is the doll that 
comes alive and enchants Hoffmann so that he falls 
in love with it/her. The ideas of artificial humans and 
simulated intelligence were also apparent at that time, 
cf. Mary Shelley’s Frankenstein? (1818) and Charles 
Babbage’s design of his analytic machine (1820s), tradi- 
tionally emphasised as the first modern, programmable 
computer. Even the word “robot” is ascribed to a theatre 
performance, the 1921 R.U.R. by Karel Capek. 

In literature, several comprehensive studies seek to 
systematise the use of new, digital media in perfor- 
mance [e.g., 2, 15]. In his comprehensive overview 
of the history of performance incorporating (digital) 
technology, Steve Dixon suggests that “the use of robots 
constitutes one of the most technologically advanced 
developments within Digital Performance” [14, p. 271]. 
The appearance of robots on stage raises the question 
of how to deal with performing artefacts that disturb 
with a fundamental understanding of live performance 
as human agency on stage. Subsequently, robots on 
stage are often discussed in terms of a broad, philo- 
sophical interest in the influence of digital technol- 
ogy on performance, in which the use of robots is 
aligned with the use of projections and other “artificial” 
onstage performers or virtual object-figures. The idea 
of (strong) artificial intelligence challenges traditional 
notions of performance, based on a notion of liveness 
that basically implies human beings on stage in front of 
a live audience. While the experience of dance, music 
and theatre as something live is valued and celebrated 
by audiences and practitioners alike, the status and 
significance of the live in contemporary performance 
is disputed [e.g., 1, 37]. For performance scholar Peter 
Echersall “a dramaturgy of robots and object-figures is 
noteworthy for the ways that it is extending the idea 
of performance” [16, p. 129], exactly because it blurs 
the borderline between live and mediated performance, 
between human and machine. One such example is the 
Robot Theatre by the Japanese duo Hirata Oriza and 
Hiroshi Ishoriguro. The main goal here is to develop 
robots that look and act as much like humans as pos- 
sible, to which end Oriza has developed what he terms 
an “actroid” — an android robot that potentially can sub- 
stitute an actor on stage, designed to act like a human 
by showing strong visual and behavioural likeness, and 
to act as if it has empathy (see, e.g., Ogawa et al [2014]). 
In this case, the autonomy or semi-autonomous actions 
are emphasised, and the focus is merely to investigate 
the technical means of production. In any experiment 
in which technological agents are used to replicate, 
augment or replace human actors in performance, the 
question of these agents’ “proto-subjectivity” becomes 


4Original text available at http://www.gutenberg.org/ebooks/41445 
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central [15, p. 113] and raises the basic question to what 
extent a robot can act? 

Another line of artistic inquiry seeks, on the contrary, 
to underline the materiality of the machinery. For 
example, the term “metal performance”, coined by 
Steve Dixon, emphasises the symbiosis of flesh and 
metal. It covers robots as well as cybernetic organisms, 
“cyborgs”, and stage the machination of the human, 
enhancing the human body through the use of, e.g., 
medical instruments, prosthetics, robotics or virtual 
reality systems. Here, the question of autonomy is put 
forward in a more distinct way, since the extensions of 
man include the ability of the technology to react in 
unforeseen ways and to operate through self-generative 
processes and not completely under the control of 
the artist, performer or programmer. The important 
question thus becomes how the technique influences 
the creative process? For both, a related question is if 
and how this autonomy or quasi subjectivity becomes 
visible to an audience and to what extent there is a need 
to create certain “visual clues” to manage the audience 
attention in a performance situation [5, 46]. See also the 
analyses of [8, 10]. 


2.2. Human Robot Interaction studies and “theatre” 
as metaphor 


The research field of Human Computer Interaction, 
HCI, considers the design and evaluation of computer 
systems, experienced through their user interface. 
Quality measures typically concern how efficiently 
and satisfactorily a human user can solve given tasks 
in an interaction with the system. Traditionally, but 
not exclusively, HCI concerns situations with a user 
seated in front of a computer system equipped with 
standard input-output devices such as high-resolution, 
graphical monitors, keyboards and computer mice. The 
user commands the system by explicit input (entering 
text, clicking buttons, etc.), and the system responds 
in various ways. Human Robot Interaction, HRI, is 
a subfield of HCI. The interaction between humans 
and robots — and thus the design and evaluation of 
such interactions — is more subtle. The relationship 
is closer to symmetric, as the human and the robot 
both may take initiatives and communicate through 
presence and action in physical space. The robot's 
communicative acts may be aided by anthropomorphic 
traits such as gestures and facial expressions, and it may 
be programmed to simulate social skills and to interpret 
the human’s spoken utterances, facial expressions and 
even body language. The interaction between human 
and artefacts may resemble a cooperative process 
between humans with a subtler and partly implicit 
communication. 

For example, [42] propose criteria for design and 
evaluation of socially assistive robots that involve 
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personality, empathy and other properties inherited 
from how we would consider a human assistant. 
This approach is also relevant for understanding less 
sophisticated and non-humanoid robots, considering 
how these artefacts are experienced (rather than what 
they are). It is, for example, not uncommon for users 
of advanced copiers to associate a certain personality 
and (lack of) empathy to these artefact, especially 
when they do not behave as expected. The conference 
paper [13] investigates the influence of a humanoid 
robot’s body language based on how it is experienced. 
The annual ACM/IEEE International Conference on 
Human-Robot Interaction provides a good overview of 
the area, and concerns roughly equal portions of reports 
of technological progress and of evaluation methods 
[see, e.g., 3, 27, 28]. 

Theatre has often been used as a metaphor in HCI and 
especially HRI for controlled environments for testing, 
having a number of “actors” (some human and some 
robots), each playing a certain “role” [e.g., 7, 25, 35, 
36, 40]. From an engineering point of view, such a 
setup has the advantage that HCI/HRI issues can be 
tested in isolation and in controlled environments, as 
the robots’ fixed manuscript eliminates the need for, 
say, convincingly simulated intelligence and ability to 
improvise. 

Such test beds are typically used in experimental 
laboratories or university settings, with the audience 
being academic staff and students and the purpose is 
to test robots as such (technical robustness and/or HRI) 
or to investigate new dramatic forms [see, e.g., 24]. Or 
it can be in school (elementary, secondary, ...) settings 
with pupils and their teachers as actors, technicians 
and audience. The purpose may be to teach theatre, 
programming, or a mix [e.g., 23]. This sort of theatre 
for testing has also been used with humans playing 
roles as robot, perhaps through remote control — which 
is a quite cost-effective way of testing new technology 
without having to actually build it first. 

Another and overlapping field is the research field 
of Social Robotics, as, e.g., manifested by an annual 
conference series, e.g., the most recent ones [38, 47], and 
a research journal; ° see also the recent collection [45]. 

In some cases, the robots’ interpretation of people’s 
doings is important for the show to succeed, in other 
experimental dance shows it is central for how the show 
proceeds. In the famous, referenced study by Ogawa 
et al. [2014], a robot with a very human-like face and 
body is citing poems to a dying woman. This actroid 
is seated in a chair, so that the need for human-like 
body movements (which are quite difficult to obtain) is 
reduced. 


Shttps://link.springer.com/journal/12369 
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In the Pas-des-deux scene of Robot! described in 
the beginning of this article, the robots’ interpretation 
of people’s doings may technically speaking be less 
important, since an illusion of this can be created by 
professional performers and (for the robot’s part) pre- 
recorded sequences. 


3. Prop or performer? 


Strong artificial intelligence, as defined by [39], i.e., 
the machine has a real mind,° is a philosophically 
intriguing idea, which is still far from present day, 
reactive technology. However, several authors have 
considered the (ethical as well as artistic) question of 
whether intelligent computers will be able to create 
(in a true sense) art [26]. Consequently it has been 
suggested that in “Robotic Art”, the robots constitute 
the work of art, rather than creating it [29]. Robots 
on stage are, thus, an inspiring area of investigation 
as robots perform alongside human performers and 
non-human props with a natural focus on the act of 
performing, rather than speculation of whether the 
robots themselves are “really” creative. 


3.1. State of the art in robot technology 


The principles of robot technology have not changed in 
any drastic way in the last few decades. The dominating 
software paradigm, called behavioural robotics, as 
formulated by [6], is based on a constantly running 
sense-decide-act loop, that at any given moment selects 
a behaviour (or combination of such) that is judged 
most optimal here and now, e.g., follow-planned- 
route, avoid-unexpected-obstacle, etc. This paradigm 
provides a simple but effective sort of autonomy, so a 
robot can perform certain tasks, while reacting suitably 
to changes in the environment. Seemingly advanced 
patterns of conduct can be induced by fairly simple 
programs. This ranges from practically relevant devices 
such as robot vacuum cleaners to the seemingly very 
robust, more or less absurd creatures exposed, e.g., by 
the company Boston Dynamics.’ Upon a behavioural 
software platform, all sorts of decision making and 
planning software can be added, as, e.g., demonstrated 
by self-driving cars, but we judge this (currently) less 
important for robots in performative contexts. 

As suggested, interesting appearances of a robot on 
stage may be achieved also without technically true 
interaction between robot and human actor, or between 
robot and audience. Pre-recorded patterns of robot 
movements and actions may be replayed by software 


®Most convincingly depicted in Stanley Kubrick’s and Arthur C. 
Clarke’s movie “2001, a Space Odyssey” as the visionary HAL 9000 
computer. 

7http://www.bostondynamics.com) 
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running on top of a behavioural platform that supports 
the robot’s physical presence, e.g., for keeping the 
balance while walking, getting back on its feet if it 
unexpectedly falls, etc. This is exemplified in Blanca 
Li’s Robot!. Robot movements may be described in 
dedicated script languages — find a recent collection of 
papers on this subject in the book “Dance Notations and 
Robot Motion” [30]. 

As we see it, the most important development in 
recent years is that robots become increasingly cheaper, 
robust and easier to program. The programmable Nao 
robot that we have mentioned several times, is a good 
example of this [19]. It comes with a fairly easy to use 
programming environment, given the illustrative name 
Choregraphe,® which is partly graphical and provides a 
large collection of predefined behaviours. It can also 
record movements, performed by a choreographer in 
slow-motion by bending the robot’s physical joints, for 
later playback. 

Another exciting development is the emergence of 
very cheap do-it-yourself hardware components, such 
as Arduino,’ by means of which excited amateurs 
without an engineering degree can tinker together 
and program their own robots, utilising whatever is 
available of cheap wheels, motors and batteries. For 
less than one hundred EURs, it is possible to produce, 
say, a box-formed, autonomously moving robot having 
a reversed broom on its top and a few blinking lamps, 
capable of reacting to an actor’s simple gestures. 

The Lego Mindstorms kits,!? introduced initially 
in 1998 and continually developed, make robot con- 
struction and programming accessible to children, and 
have had the effect of de-mystifying the understanding 
of what is a robot:!! it is not necessarily associated 
with true and dangerous “artificial intelligence”, but 
is typically a simple mechanical artefact controlled by 
straightforward algorithms. 


3.2. Robots as performative gestalts 


The reception of robots depends very much on the 
expectations of the audience as it is imposed by the 
context of the performance. The studies mentioned 
sofar most often assume a more or less humanoid 
robot appearance as this may naturally enhance 
humans’ interpretation of robot behaviour. Recent 
research concerns another scenically interesting and 
challenging issue of making non-humanoid robots 
expressive, related to a study of the receiver’s cognitive 
mechanisms [see, e.g., 4, 17, 22, 31-33, 41]. 


8See http://doc.aldebaran.com for details. 

°https://www.arduino.cc 
!0https://www.lego.com/en-us/mindstorms 

11 de-mystifying, as opposed to the way robots are often portrayed 
in movies and journalistic writings. 
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For example, Gemeinboeck and Saunders [18] have 
experimented using machine learning techniques to 
identify expressive patterns of movements, created 
initially by human performers placed inside flexible 
“artefactoid” objects. Takayama et al [41] have applied 
results from studies of animations to make the 
intentions and reactions of a robot in social contexts 
easier to interpret by humans. A variant of this is the 
study of robots appearing as familiar domestic objects, 
FDO Robots [11], which in addition to expressive 
movements, also takes into account the audience’s 
expectations of contexts and situations associated with 
the chosen FDO. FDO robots in performances also 
emphasise the importance of the human co-actors’ 
pretending an emotional contact with the robot, thus 
creating an illusion of the robot as a creature having 
emotions; see Figure 2 for an illustration of this. See 
also [43] for another appearance of FDO robots in the 
shape of animated furniture. 


Figure 2. Human actor in close and emphatic contact with an 
FDO robot, whose body is borrowed from an old-fashioned vacuum 
cleaner. 

(Photo copyright © Mads Hobye 2018) 


As suggested in Section 2, there is a variety of 
situations in which robots appear on stage, ranging 
from full scale, professional performances for a paying 
audience over short (entertaining) intermezzi in larger 
events such as concerts and TV shows, to experimental 
laboratory or university settings with its audience 
being academic staff and students [see, e.g., 24] or 
performance installations in which there is no clear 
distinction between auditorium and stage [see, e.g., 44] 
and in which the interaction between performers and 
audiences is continuously negotiated. 

Demers [12] coins the term “machine performers” 
in order to cover a broad spectrum of inanimate 
performing artefacts. From a phenomenological and 
embodied perspective on his own work The Tiller Girls, 
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he emphasises how also machine performers need the 
co-presence of the audience to be fully realised (p. 276). 
In other words, in robot performance as it appears for 
the audience in the auditorium, it is not necessarily the 
question of autonomy that goes first. 


4. Five Dimensions to Describe Robots on Stage 
as Performative Gestalts 


Robot! stages, as stated earlier, the relation between 
humans and modern technology as a continuum 
allowing for humans acting as machines and machines 
becoming animated. The show includes different 
notions of a robot in each of its sequences. Besides 
the use of Nao robots (that appear approximately 
15 minutes out the show’s 60 minutes duration) 
and human dressed like and/or acting robot-like, the 
show includes among others a self-moving litterbin 
on wheels, remote controlled play toy tanks, vacuum 
cleaners and an on-stage orchestra made out of humans 
sized mechanical instruments. 

Following the idea of machine performers calls for 
a common interest in these robot as performative 
gestalts. This includes their expressive as well as 
anthropomorphic character of their robotic gestalta, 
and it gives special attention to questions like: What 
are the roles of the robots used on stage — do they play 
a main character or is it just a supernumerary? Is it a 
robot playing a human or does it play a role as a robot 
and by which means are human-like, communicative 
features created? Is there one robot or are there many? 
Do they interact with one another or with human 
dancers? Some of the answers to these questions lie in 
the making of the robot — its level of mobility, its looks 
and gestures and the use of exaggerated clues. Others 
are to be found in the role the robot is playing, in its 
interaction with other figures on stage and in the way 
in which an audience is invited to react with empathy 
or even to project human feelings onto the machine 
performer. In the case of Robot!, the physiognomy of 
the Nao, with its human baby body shape, its limbs 
and blinking eyes, matters, but what makes it most 
believable is the way in which the human dancer treats 
the Nao and reacts to its movements and gestures. 

Below we introduce five dimensions suited for char- 
acterising the robots on stage as performative gestalts. 
These dimensions describe a metaphorical vector-space 
in which each point indicates one particular form of 
appearance of “a robot”. The other way round, measur- 
ing these dimensions of a specific staged robotic appear- 
ances serves as a tool to describe it with special regard 
to its expressivity potential and anthropomorphic fea- 
tures. Our choice of dimensions condense the mate- 
rial presented in the previous sections inspired by the 
questions raised by the robotic appearances in Robot! 
and the various performances listed in Section 2.1. The 
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particular number of dimensions (and the definitions 
thereof) are not final, but open to elaboration and 
discussion. To illustrate each dimension, we visualise its 
potential value space as an indicative linear scale that 
orders some selected values. Of course, this can only be 
a coarse approximation, since our dimensions in most 
cases contain more complex values than can be ordered 
in a linear way. 


Dimension 1. Movements 


The ability to move is often seen as a defining 
characteristic feature of an artefact called a robot. On 
stage, movements are among the prime instruments 
to create an impression of agency, independently of 
whether the robot changes its position, moves its parts 
or both. Absence of movements, or limited movement 
may also be used dramaturgically. 


In Robot!, the central Nao robot is demonstrated 
with its many degrees of freedom in its creaky 
movements: it is dancing in an often clumsy way, 
it walks, falls occasionally and is able to rise again. 
In several sequences, most notably in the Pas-des- 
deux, Nao’s clumsy movements, small size and doll-like 
appearance give the impression of a child, appealing 
to the audience’s sympathy. A quite different robot in 
Robot ! is a self-moving litter bin on wheels, opening and 
closing its lid, creating the impression of a mouth ready 
to eat the litter it is served. 


The classical industry robot arm can produce 
very complex movements, very fast and with high 
precision; best known is the KUKA mentioned 
above from 1973 that can rotate around 6 axes. 
As it has been demonstrated in numerous shows, 
performances, interactive installations, etc., these robot 
arms can produce very suggestive human-like dance- 
like patterns, even though they are typically fixed to the 
floor and do not at all look like a human dancer. This is 
demonstrated convincingly in the video!” from Mylene 
Farmer’s Concert Tour in 2013 mentioned above. 


Flying drones or quadcopters can move in three 
dimensions and tilt or rotate in different ways (called 
roll, pitch and yaw) with high flexibility and accuracy, 
although not with the speed of a KUKA robot arm. 
The lampshades in the Sparked video!’ by Cirque du 
Soleil are made flying by such drones (invisibly to the 
spectator), and they can react instantly to a human 
actor’s movements, thus creating the impression of a 
dialogue by movements alone. 


!2https://www.youtube.com/watch?v=wQ3MqPUqCes 
'Shttps://www.youtube.com/watch?v=6C8O0JsHfmpl 
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Dimension 1, movements. Indicative scale: 
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In this case, the simplification into a linear scale blurs 
the distinction between a stationary robot moving its 
parts (say, a KUKA robot arm), and a robot able to 
traverse the whole stage space in three dimensions (say, 
a flying lampshade); both could classify as “flexible, fast 
and agile”. The two middle samples may represent, say, 
a consumer vacuum cleaner and the Nao robot. 


Dimension 2. Apparent level of autonomy and 
interaction 


While the previous dimension is close to being measur- 
able in terms of numbers, the dimension of autonomy 
and interaction relies on qualitative parametres. As we 
have discussed, the term “apparent” is important, as 
a well-planned bluff, perhaps supported by competent 
human performers, may be at least as convincing as 
relying on current AI technologies. 

A robot may behave in a highly mechanistic way, 
always giving the same response in similar situations, 
or it may appear as an active agent that takes its own 
decisions, or at least showing immediate and perhaps 
spontaneous reactions. '* 

The Pas-des-deux scene of Robot! is an eminent 
example of how a meticulously developed script, 
executed with accurate precision by a robot, is part of 
creating a perfect illusion of an autonomous entity that 
interacts according to its emotional state. 

We see interaction and communication tightly 
integrated with autonomy, as any communicative act 
reflects a decision of doing that act. Reactivity is a 
related term: the act of avoiding an obstacle, say, a 
human, rather than bumping into it, may also be seen 
as a communicative act, expressing an intention of not 
harming that obstacle. On the other hand, reactivity is 
typically applied to mechanistic reactions to changes in 
its context (cf. its antonym proactivity). 

The consumer vacuum cleaner robot that appears 
in the tumultuous final scene of Robot! together with 
numerous other mechanised artefacts is — technically 
speaking - the only fully autonomous robot in 
that performance. However, there is no dramaturgic 
emphasis on this autonomy, and it is anyhow in the part 


'4Tt is a well-known trick among programmers of robots and 
computer controlled artistic installations, that adding a bit 
randomness and noise — i.e., programmed imperfection — is an easy 
way to obtain some impression of humanness and intelligence [our 
own observation; no documentation]. 


EAI Endorsed Transactions on 
Creative Technologies 
06 2020 - 12 2020 | Volume 7 | Issue 25 | e5 


Henning Christiansen, Anja Mølle Lindelof 


of the spectrum for mechanistically, reactive robots. We 
may hypothesise that the purpose of including it in 
Robot! is exactly to demonstrate this sort of dumb but 
useful robots. 

Robot Soccer represents an ultimate exposition of 
genuine robot autonomy, interaction and coordination. 
As in human soccer, the unexpected and known- 
to-be-unknown final outcome creates an additional 
excitement. Robot soccer competitions are in most cases 
organised as part of technical rather than artistic or 
scenic events; however, they are experienced by the 
audience as entertaining and engaging performances.!° 


Dimension 2, apparent autonomy. Indicative scale: 
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A stance in between the middle and right examples 
in the scale is represented by the dance performance 
School of Moon from 2016, choreographed by Fric 
Minh Cuong Castaing. Here children and Nao robots 
are imitating each other’s movements, guided by two 
professional dancers. The result is a very fine and 
subtle communication between humans and robots, 
that seems difficult to create by the bluffing approach 
we have praised so far.!° This also points towards the 
possibilities of different performance genres and the 
relative ease with which, e.g, improvisational theatre 
and experimental dance are able to adapt and explore 
autonomous performative gestalts in the performance 
situation. 


Dimension 3. Appearance: humanoid, abstract or 
perhaps something else 


The perception and reception of an entity on stage as a 
robot is a combination of different and often coherent 
properties. For example, the Nao’s visual appearance as 
a small plastic doll with slightly oversized head (this 
dimension) together with its clumsy way of moving 
(Dimension 1) may likely be experienced as resembling 
a child. In order to keep our dimensions more or 
less independent, Dimension 3 may be though of as 
what can be deduced from still images of the entity in 
question. 

We have already described several different robots 
of Robot!, including some that are played by human 


'SSee the official RoboCup webpages http://www.robocup.org for 
background and links to a videos. 

'6Rind more information and videos at 
http://shonen.info/schoolofmoon/. 
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performers. Some robots appear very human-like, such 
as Hiroshi Ishiguro’s seated lady reciting poems in 
Mirata’s “Sayonara” and his showcase of a robot looking 
exactly as himself. In the other end of the spectrum, we 
find totally abstract objects used mostly in academic 
studies such as [18] for which only their movements 
indicate some sort of agency. An extreme example 
of such an object may be a box or sphere with 
no texture at all and a colour of 18% gray. Robots 
dubbed as “humaniod” come in many variations: the 
mentioned “very human-like” ones, Nao’s abstract (but 
dramaturgically effective) human-like body, strange 
devices moving on wheels with the association to a 
human are created by an animated face shown on 
a computer screen placed at the top of the device 
as exercised in several works by Stelarc [20, 21]. An 
effective and minimalist way to make an artefact give 
association to something human or an animal is giving 
it (perhaps a very abstract representation of) eyes: two 
LED lamps placed in a suitable distance on a robot 
arm are sufficient, as, e.g., appearing in the video from 
Mylene Farmers Concert Tour mentioned several times. 

Robots may also appear in the shape of familiar 
domestic objects, say a chair, a table or an old-fashioned 
manual vacuum cleaner, as already mentioned [11]; 
there is nothing “robotic” in their visual appearance, 
but as soon as they start acting, this changes. The 
visual design of mechanical instruments delivering 
music throughout most of the Robot ! performance takes 
another direction: a relationship to humans is created 
by the size of the instruments and an occasional hand 
or pair of lips, but apart from that, they take one 
form more absurd that the other, something in between 
creatures from a painting by Hieronymus Bosch and 
selected rock LP covers of the 1960s. 

Overlapping with the previous category are house- 
hold robots such as the typically flat round consumer 
robot vacuum cleaners and robot lawn movers, which 
can appear as themselves, triggering a sort of famil- 
iarity. The industrial robot arm is not truly household 
and most people have never been close to one, but will 
anyhow likely recognise it, since robots arms have been 
around for more than half a century and have become a 
standard icon for modern industrial production meth- 
ods. 

We will also mention human performers — recogniz- 
able as such — taking roles of robots. In Robot !, we see 
an ensemble of humans dressed in colourful cardboard 
box costumes representing a notion of a robot known 
from comics series and children’s drawings, and we see 
normal looking people whose robotic nature is indi- 
cated by exaggerated stiff and “algorithmic” movements 
(i.e., a value in Dimension 1). An intriguing example 
of a human acting as a robot is the already mentioned, 
Olympia of The Tales of Hoffmann that is multilayered in 
the sense that the audience is expected to recognise that 


8 EAI Endorsed Transactions on 


Creative Technologies 
06 2020 - 12 2020 | Volume 7 | Issue 25 | e5 


Robots on stage 


a human singer plays the role of a robot playing the role 
as a human; the robotic layer be may emphasised by the 
costume (this dimension) — Olympia is often equipped 
with a winder — and is exercising stiff movements 
(Dimension 1). The options for visual appearance are 
endless, and let us end with an extreme example of a 
robot having no visual body - or rather, the entire stage 
space is its body; HAL in Kubrick and Clarke’s movie 
“2001: A Space Odyssey” is a frightening example of 
this. 

It is impossible to compress the rich space of visual 
robot appearances into a meaningful linear scale, so 
instead we show a line corresponding to “degree of 
humanoidism” and let those that do not fit in here float 
in the space, separate from the line. 


Dimension 3, appearance. Indicative scale: 
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Dimension 4. Massification or individual 


Massification is a well-known and recognised parame- 
ter in performances and other art forms, and is worth 
emphasising for robots as well. This goes beyond basic 
questions such as if there is one or many or if they play 
a main character, supporting roles or act as supernu- 
meraries. Of course, such traditional terms may apply 
to robots as well. More interesting to pursue is the dra- 
maturgic potential of robot technology’s intrinsic possi- 
bility of mass production of totally identical robots, per- 
haps executing the exact same movements at exactly the 
same time. Since ancient Greek dramas, the chorus has 
often played the role of the public opinion; similarly, 
a chorus consisting of robots rather than people may 
be chosen to emphasize a mechanistic-stupid pattern 
of reaction. Identical or almost parallel movements of 
a few, say two to five, or up to many similar per- 
formers create a strong visual effect of parallelism and 
repetition, emphasising the characteristics of specific 
movements. This is used in several robot performances 
such as the (partly) synchronised drones dressed up as 
flying lampshades in Sparked by Cirque du Soleil, and 
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the 32 small robots in Demer’s (2016) The Tiller Girls. In 
Robot !, following the Pas-des-deux scene, four more Nao 
robots come on stage, facing the audience, and all five 
strictly identically looking robots attempt to perform 
the same synchronised dance. Now and then one of 
them falls, gets up again and finds its way back in line. 

Massification also comes in another, apparently less 
structured way, in which a large number of actors create 
different patterns of movements. In Robot! this happens 
in the final sequence, where a horde of mismatched 
robots appears; Naos, a consumer vacuum cleaner; 
strange, remote controlled wheeled boxes and the 
mobile litter bin, opening and closing its lid as a mouth. 
Robots are here accompanied human actors, speeding 
around like stressed robots suffering from some sort 
of programming error. There is no synchronization 
or parallel movements, but massification nonetheless, 
creating an expressive chaos. 

In the other end of the spectrum, the Nao robot 
appears as protagonist in other sequences of Robot!, 
relying on its special characteristics (especially charm) 
that makes it different from other pre-fabricated robot 
brands on the market; for example, it tries itself out 
as a crooner. In the Pas-de-Deux scene that we have 
mentioned, it appears as one of two equal partner, and 
the same is the case in another sequence, in which Nao 
performs a clumsy and failed attempt to seduce an 
alluring female dancer. 


Dimension 4, Massif. or indiv. Indicative scale: 
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As shown, we indicate the degree of massification by 
counting with an addition of to which extent a group 
of robots are coordinated. When characterising this 
dimension for a given robot, we may also expect an 
indication of the types of other robots together with 
which it appears; this is done in Section 4.1 below. 


Dimension 5. Sound, including noise, speech and 
singing 

The last dimension we address is sound. For any 
show, sound plays a constituting role, and this ranges 
from proper dialogue and singing, over expressive non- 
verbal sounds to the planned as well as inherent sounds 
of the machinery, combined in a show’s soundscape. 

In Robot! - as in many dance performances - 
speech plays only a very small role. Instead it 
has a continuously changing musical soundscape, 
partly through loudspeakers and partly by an on- 
stage orchestra made up by human sized mechanical 
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instruments, whose visual appearance was described 
under Dimension 3 above. They produce perfectly 
timed techno-like music relying on mechanical and not 
synthesizer sound (which makes quite a difference). 


While convincing human-like speech is difficult to 
produce by a computer, it is easy to include recorded 
sound. When Nao acts as a crooner, loudspeakers 
transmit the Spanish love song “Besame Mucho” as sung 
a human, deep timbre, sensual female voice. The closest 
we get to speech in Robot! is in the failed seduction 
scene (see Dimension 4, above) in which the female 
dancer utters a sort of abstract language composed of 
words and phrases in different languages and indicative 
sounds, and Nao replies in a similar way. 


Robots and other moving artefacts produce a varying 
amount of mechanical noise, which may be seen as a 
problem or may be emphasised and even artificially 
amplified. In Robot!, the sound of five dancing Naos’ 
creaking joints blends into the soundscape. The litter 
bin mentioned already emits a very distinct metallic 
“clang” each time its lid closes. 


We may suggest the examples of sounds mentioned 
above order into a scale as follows. 


Dimension 5, sound. Indicative scale: 
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The litter bin’s “clang” is an example of what we call a 
nonverbal communicative utterance. 


4.1. Exemplifying the Dimensions as Descriptive and 
Design Tool 


As mentioned in the introduction to this paper, the 
purpose of our chosen dimensions is two-fold, they 
should be applicable as a research tool to characterise 
an existing performative gestalt and as a design tool 
for novel appearances. Spoken with a mathematical 
simplicity, both tasks are straightforward: we just have 
to fill in the five coordinates with values that identify 
the specific point in 5-dimensional space; done! 

We test the dimensions first as a descriptive tool in 
an attempt to characterise the litter bin of Robot!, that 
has been mentioned several times above. We suggest the 
following five coordinate values. 
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Moves its body slowly forward, 
changes direction more or less 
randomly; opens and closes lid. 
Simple responsive, but from time 
to time experienced a bit cheeky. 
A Familiar Domestic Object: a 
medium-sized pedal bin of the 
sort you can buy almost every- 
where. 

Mainly individual; occasionally 
flocking with cleaning workers, 
and in the final sequence with lots 
of diverse robots. 

A metallic, repeated “clang- 
clang-clang-...” produced by the 
lid being closed. 


1: | Movements 


2; | App. autonomy 
and ... 


3: | Appearance 


4: | Massif. or indiv. 


5: | Sound 


This schematic presentation is concise and factual, but 
it also displays the important anthropomorphic traits of 
the object in focus. For example, there is an immediate 
interpretation of what is the front and back of it (that 
is why we could write above that it moves forward), 
and a glimpse of a certain personality associated with 
it appears when it is characterised as “cheeky”. Writing 
down such a description of every instance of the 
phenomenon robot that appears in Robot ! will be a good 
starting point for any thorough and deep analysis of this 
performance. 

Now we make a though experiment of using the 
dimensions as a design tool. Assume we are involved 
in the setting of Johann Strauss’ operetta Die Fledemaus, 
and we get the idea of animating a central scene 
object, namely prison guard Frosch’s Slivovitz bottle. 
Frosch is a spoken role, quite often performed by some 
renowned actor of a certain age with a large element of 
improvisation; it is very humorous scene, perhaps with 
a tragic undertone due to Frosch’s obvious alcoholism. 

Our vision of an animated Slivovitz bottle has a 
personality that wants to tease and quarrel, but is 
basically friendly. As a first draft of a design, we fill in 
the five dimensions as follows. 
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Can move back and forth on 
a specific straight line on a 
table, moving relatively fast and 
accelerating very quickly; may 
occasionally be lifted from the 
table by a human actor and put 
back on the same line. 

It reacts spontaneously and 
instantly to an approaching 
human hand, sometimes moving 
away playing hard to get and 
sometimes allowing the hand to 
lift it. 

A Familiar Domestic Object: a 
bottle labelled “Slivovitz” con- 
taining a diminishing amount of 
a yellowish-brownish liquid. 

An individual that is fully aware 
of its own precious value. 
Occasionally a sound of sloshing 
liquid (triggered by the human 
actor’s handling of the object). 


1: | Movements 


2. | App. autonomy 
and ... 


3: | Appearance 


4: | Massif. or indiv. 


5: | Sound 


From this description it is possible to make a first 
evaluation of the envisioned animated object. A 
dialogue between actor and technicians may concluded 
that this is “difficult but doable; if we can make it work, 
it may create a sensation, so let’s give in a try”. 


5. Conclusion 


Inspired by Blanca Li’s Robot !, this article investigates 
the appearance of robots on stage in contemporary 
performance. As a convincing demonstration of the 
relevance of robots — as artefact, concept, metaphor 
- this performance unrolls a series of different 
instantiations and situations of staged robots. 

On this basis, we have proposed a classification 
matrix of five dimensions which may provide vocabu- 
lary and terminology to characterise different instances 
of robots on stage with a focus on their expressive char- 
acter as performative gestalts. Accordingly, this means 
that we focus on the intended audience experience 
rather than on how this experience is produced by 
technological means. Therefore we include the notion 
of a robot in a wide sense, ranging from advanced 
technological artefacts, over simple objects that display 
some sort of reactiveness or initiative-taking. 

Beside references to Robot!, the dimensions refer to 
main developments and discussions regarding, e.g., the 
level of autonomy, mobility and questions of humanoid- 
ization in performance studies and HRI. Such questions 
highlight potential dilemmas in the study of robots, 
between the fascinating, autonomous behaviour of 
robots and the demands of control in show business. 

The five dimensions of the matrix — Movements, 
Apparent level of Autonomy & Interaction, Appear- 
ance, Massification and Sound - represent essential, 
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identifiable properties which are more or less orthog- 
onal and that span a rich and complete space of 
robotic performative opportunities, focusing on robots 
as expressive gestalts. These dimension indicate a 
metaphorical vector-space, in which each point indi- 
cates one particular form of appearance of a robot on 
stage. 

The applications of the dimensions can serve as an 
analytical tool investigating the use of robots across 
performance genres. More importantly, we see the 
dimensions as a template that may inspire the design 
of new ideas. Cheaper and easy-to-use technology 
enables artistic use of robotics in a lot of settings and 
allows for interdisciplinary collaborations and small- 
scale students experiments. Ideally, the matrix becomes 
useful as a tool for researchers, technical developers 
and performance practitioners to better understand 
and to invent new ways of including robots in an 
interdisciplinary collaboration between. 
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